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Abstract 

In this paper, we give a necessary and sufficient condition for mean stability of switched linear systems having a Markov 
regenerative process as its switching signal. This class of switched linear systems, which we call Markov regenerative switched 
linear systems, contains Markov jump linear systems and semi-Markov jump linear systems as special cases. We show that a 
Markov regenerative switched linear system is mth mean stable if and only if a particular matrix is Schur stable, under the 
assumption that either m is even or the system is positive. 
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1 Introduction 

Among switched linear systems, those having a time- 
homogeneous Markov process as its switching signal, 
called Markov jump linear systems (Costa, Fragoso & 
Todorov 2013), are of particular importance. One of the 
reasons for their importance is that time-homogeneous 
Markov processes are well suited to model stochastic 
phenomena presenting a constant rate of occurrence. An¬ 
other reason is that the analysis and synthesis of Markov 
jump linear systems can be performed in a rather simi¬ 
lar way to those for linear time-invariant systems by the 
introduction of auxiliary variables (Costa et al. 2013). 

However, it is often restricting to assume that the switch¬ 
ing signals present constant transition rates, or even 
the Markovian property. To overcome this restriction, 
we find a wide variety of alternative switching signals 
in the literature (Hou, Luo & Shi 2005, Antunes, Hes- 
panha & Silvestre 2013, Ogura & Martin 2015, Ogura & 
Martin 2014). A natural extension to time-homogeneous 
Markov processes are time-homogeneous semi-Markov 
processes, which are Markovian-like processes with time- 
varying transition rates (see ((linlar 1975). The stability 
analysis of the corresponding switched linear systems 
can be found in Antunes et al. (2013) and Ogura & Mar¬ 
tin (2014). Another extension are regenerative processes 
(Smith 1955), which are, roughly speaking, stochastic 
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processes that can be obtained by concatenating inde¬ 
pendent and identically distributed random functions 
(see Sigman & Wolff (1993) for the details). The mean 
stability analysis of linear systems subject to regenera¬ 
tive switchings is performed in Ogura & Martin (2015). 

In this paper, we extend the works in Antunes et al. 
(2013), Ogura & Martin (2015), and Ogura & Mar¬ 
tin (2014) to analyze the stability of Markov regener¬ 
ative switched linear systems, which are switched lin¬ 
ear systems whose switching signal is a Markov regen¬ 
erative process (also called a semi-regenerative process) 
(^inlar 1975, Choi, Kulkarni & Trivedi 1994). Markov 
regenerative processes form a large class of stochastic 
processes which contains as special cases all the Markov, 
semi-Markov, and regenerative processes. We show that 
exponential mth mean stability of a Markov regenerative 
switched linear system is characterized by the spectral 
radius of a matrix, under the assumption that either m is 
even or the system is positive. Extending various results 
in the literature (Antunes et al. 2013, Ogura & Martin 
2015, Ogura & Martin 2014, Fang & Loparo 2002), the 
obtained result enables us to analyze the stability of, for 
example, state-feedbacked Markov jump linear systems 
with periodically observed mode signals (see a discrete¬ 
time setting in Cetinkaya & Hayakawa (2015)), as well 
as controlled system with failure-prone controllers with 
only one repairing facility (Distefano & Trivedi 2013, 
Section 3.2). 

The paper is organized as follows. After introducing nec¬ 
essary notations, in Section 2 we introduce the class of 
Markov regenerative switched linear systems under con¬ 
sideration and then state a necessary and sufficient con- 
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dition for their exponential mean stability. Then, in Sec¬ 
tion 3, we present various applications of the main re¬ 
sult. The notation used in this paper is standard. When 
X G M" is nonnegative entrywise we write a: > 0. The 
standard Euclidean norm on M" is denoted by H-H. Let 
/ and O denote the identity and zero matrices, respec¬ 
tively. The block diagonal matrix with block diagonals 
Ai, ..., Aat is denoted by Ai. The kronecker prod¬ 
uct of two matrices A and B is denoted hy A® B. We 
say that A G is Schur stable if A has the spectral 

radius less than one. Also we say that A is Hurwitz sta¬ 
ble if all the eigenvalues of A have negative real parts. 
For an integrable random variable A, its expected value 
is denoted by E[X] and its conditional expectations by 
E[X I •]. 

2 Stability characterization 

This section introduces the class of Markov regenera¬ 
tive switched linear systems and then presents a nec¬ 
essary and sufficient condition for their stability. We 
need to first recall the definition of Markov renewal pro¬ 
cesses (Qinlar 1975). Let A be a positive integer. A 
stochastic process (0 ,t) = {{9k,Tk)}k>o taking values 
in {1,..., A} X [0, oo) and satisfying 0 = tq < n < • • • 
is called a Markov renewal process (Qinlar 1975) if 

B(^0k-\-l — j) '^k+l '^k 'E f I 9k-, . . . , 9q, Tk, ... 5 To) 

= P{9k+1 = j, Tk+l - Tk <t\9k) 

holds for every k and t > 0. We assume that {9,t) is 
time-homogeneous, that is, for all i,j and t > 0, the 
probability P{9k+i = j,Tk+i - Tk < t \ 9k = i) is in¬ 
dependent of k. We note that, in this case, 0 is a time- 
homogeneous Markov chain and therefore has the con¬ 
stant transition probabilities P{9k+i = j \ 9k = i)- In 
this paper, we furthermore assume that 

Tfe+i - Tfc > 0 (1) 

with probability one for every k. Then, we can state the 
definition of Markov regenerative processes as follows. 

Definition 1 (Qinlar (1975), Choi et al. (1994)) 

Let a = {(7t\t>o be a stochastic process taking values in 
a finite set A. We say that a is Markov regenerative if 
there exists a Markov renewal process {9, r) taking val¬ 
ues in {1,..., N} X [0, oo) and satisfying the following 
conditions for every k: 

(PI) Tk is a stopping time for a; 

(P2) There exists a function tt: A —)■ {1,..., N} such 
that T{(Trk) = 9k; 

(PS) For j e {1,... ,N}, i > 1, 0 < ti < t 2 < ■ ■ ■ < ti, 
and a function /: A^ —)■ [0, oo), 

E[f {o'r,.+ti, ■ ■ ■ t e^Tk+te) I 9k = j, 0's', S ^ Tfc] 

= Elf(ot,,...,Ot,) I 9o =j]. 


We call T the regeneration times of a, and {9, t ) the 
embedded Markov renewal process ofo. 

Among the three conditions in the definition, (P3) is the 
most important. It implies that, as far as prediction is 
concerned, at the regeneration time Tk, the past infor¬ 
mation of the process {<Ts}s<rfc is irrelevant and only the 
value of 9k is needed. Actually, (P3) implies that 

P{Or^+t = >^\ 9k = j,Os',S < Tk) = P{ot = A I 6*0 = j) 

( 2 ) 

for all A e A, z, j e {1,..., N}, t >0, and k > 0. Equa¬ 
tion (2) indicates that, given 9k = j, the process cr “re¬ 
generates” at time Tk as if it starts from time 0, given 
9o = j- Moreover, o possesses a certain Markovian prop¬ 
erty at the regeneration time r^; once we know the value 
of Ork, using (P2) we can determine the value 9k, which 
then determines the future distribution of o by (2). As 
is pointed out in ()linlar (1974), it is convenient to take tt 
to be the identity, but this is not necessary. Also we re¬ 
mark that (PI) is a technical condition needed to state 
(P3). For the details, the readers are referred to Qinlar 
(1975) and Qinlar (1974). 

Then, we introduce the class of switched linear systems 
studied in this paper. 

Definition 2 Let o be a Markov regenerative process 
and let Ax G for each X G A. Then, the stochastic 

differential equation 

(3) 

where x(0) = xq G K” is a constant, is called a Markov 
regenerative switched linear system. 

For example, a time-homogeneous Markov process r 
is Markov regenerative with an underlying embedded 
Markov renewal process being {{rt,,,tk)}k>o, where 
0 = to < fi < ^2 < • • • are the times at which the pro¬ 
cess r changes its value. Therefore, Markov jump linear 
systems (Costa et al. 2013) are Markov regenerative 
switched linear systems. In Section 3, we will show in de¬ 
tail that more general classes of switched linear systems, 
such as semi-Markov jump linear systems (Antunes 
et al. 2013, Ogura & Martin 2014) and regenerative 
switched linear systems (Ogura & Martin 2015), are 
contained in the class of Markov regenerative switched 
linear systems. 

Based on the embedded Markov renewal process, we 
can naturally define the stability of Markov regenerative 
switched linear systems as follows. 

Definition 3 Given a positive integer m, we say that S 
is exponentially mth mean stable if there exist C > 0 
andp > 0 such thatE[\\x{t)\\"'] < Ce~^*\\xo\\'^ for every 
Xq and 9q . 
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Also, we here introduce the positivity of S. 

Definition 4 We say thatY, is positive ifxo > 0 implies 
x{t) > 0 with probability one for all Oq and t > 0. 

We remark that, for E to be positive, it is clearly suf¬ 
ficient that all the matrices A}, (A € A) are Metzler, 
i.e., the off-diagonal entries of each A\ are all nonneg¬ 
ative (Farina & Rinaldi 2000). However, this sufficient 
condition is not necessary as is shown in Ogura & Mar¬ 
tin (2015, Example 10) for regenerative switched linear 
systems. 

In order to state the main result of this paper, we recall 
the notion of induced matrices. First we define (see, e.g., 
Parrilo & Jadbabaie 2008) the m-lift of a; € M", denoted 
by xl™!, as the real vector of length 
with its elements being the lexicographically ordered 
monomials \/alx°‘ (a! := m!/(Q;i! • • • a„!)) that are in¬ 
dexed by all the possible exponents a = (ai,... ,a„) 
summing up to m. For example, for x = [xi,X 2 ]' 
we have = x, = [xf,-^ 2 x 1 X 2 , and 

xA\ = [xf, •\/3 xiX 2,-v/Sxixl, xf]^. Then, we define the 
mth induced matrix of A, denoted by A^'^f as the 
Tim X Tim Unique matrix (Parrilo & Jadbabaie 2008) 
satisfying (Ax)!™! = for every x G K". 

The next theorem is the main result of this paper. 

Theorem 5 Let pij = P{9i = j \ Oq = i) for all 
i,j G {1,...,A^}. For all 0 <s<t< oo, define the 
W^^'^-valued random variable $(f;s) by the differential 
equation d^/dt = A^^<^{t\s) with the initial conditions 
^{t;t) = In for every t > 0. Assume that the following 
two conditions hold: 

(Al) Either m is even orY is positive; 

(A2) There exists T > 0 such that Tk+i — Tk < T with 
probability one for every k > 0. 

Then, Y is exponentially mth mean stable if and only if 
the (Num) X (Num) Teal block matrix A = [Aij]i<ij<]s[ 
with the {i,j)-block Aij being defined by 

A, = PjA[^A ■ 0)[”^l I do = J, Oi = i] (4) 
is Schur stable. 

In Section 3, we will show that the above theorem can re¬ 
cover stability characterizations of Markov jump linear 
systems (Fang & Loparo 2002, Ogura & Martin 2014), 
semi-Markov jump linear systems (Antunes et al. 2013, 
Ogura & Martin 2014), and regenerative switched lin¬ 
ear systems (Ogura & Martin 2015). We will also show 
that the theorem can give a stability condition for other 
switched linear systems whose stability cannot be stud¬ 
ied using the methods in the literature. 


2.1 Proof 

We present a proof of Theorem 5 in this subsection. This 
proof consists of the following two steps. We first show, 
in Proposition 6, that the stability of Y can be analyzed 
based on its discretized version. The proof of the propo¬ 
sition utilizes the technique developed in Ogura & Mar¬ 
tin (2015) to study regenerative switched linear systems. 
To analyze the discretized version of the system, we then 
present Proposition 7, which extends Theorem 3.4 in 
Ogura & Martin (2014) for not necessarily positive sys¬ 
tems. 

Let E be a Markov regenerative switched linear sys¬ 
tem. Then, the discretized process Xd = {x{Tk)}k>o is 
the solution of the discrete-time system SY : Xdfk + 
1) = ^iTk+i;Tk)xd{k). In order to proceed, we intro¬ 
duce a class of switched linear systems called discrete¬ 
time semi-Markov jump linear systems (Ogura & Martin 
2014). Let be a stochastic process taking val¬ 

ues in M”^”. The system E^ : Xd{k -I- 1) = FkXd(k) 
(X(i(0) = xo) is said to be a discrete-time semi-Markov 
jump linear system if there exists a time-homogeneous 
Markov chain 6 taking values in {1,..., A^} such that, 
for all k > 0, i,j G {1,..., A^}, and a Borel subset G of 
I^nxn, holds that 

P{0k+i =j,FkGG\0k,..., 00. Fk-i. ■■■.Fo) 

= P{0k+i=j,Fk&G\0k), 

and the conditional probability 

P{0k+i =j,FkGG\0k= z) (6) 

does not depend on k. The mean stability and the pos¬ 
itivity of Yd is defined in the following standard man¬ 
ner. For a positive integer m, we say that E^ is exponen¬ 
tially mth mean stable if there exist C > 0 and /3 > 0 
such that A[||xd(fc)||'"] < C'e“^^||xo||'" for all xq and do- 
Also we say that E^ is stochastically mth mean stable if 
J2T=o -®[ll^d(^)ll™] finite for all Xq and dg- Finally, Yd 
is said to be positive if xq > 0 implies Xd(k) > 0 with 
probability one for every k and dg. 

The next proposition relates the stability of E and SY. 

Proposition 6 SY is a discrete-time semi-Markov 
jump linear system. Moreover, if (A2) holds, then the 
following statements are true: 

• IfYis exponentially mth mean stable, then SY is 
stochastically mth mean stable. 

• IfSY is exponentially mth mean stable, then soisY. 


PROOF. Let Fk = ^{Tk+i]Tk). Also, we let (H, M, P) 
denote the underlying probability space. We denote the 
CT-algebra of O generated by a set X of random variables 
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by M{X). We define the following cr-algebras; A4i = 
M({0k}),M2 =M({0k,--.,0o,Fk-i,...,Fo}),M3 = 
M({dk, ■ ■ ■ < Tfe}), and M 4 = M({0k,f^s;s < 

Tfe}). Then, we can show 

MiCM2CM3=M4. ( 7 ) 

The hrst inclusion is obvious. The second inclusion is 
true because Fq, ..., Fk-i are measurable on M 3 . Fi¬ 
nally, the last identity follows from (P2). Now, from (P3) 
we know that P{9k+i = j,Fk G G \ Mi) = P{0k+i = 
j, Fk G G \ M 4 ). Therefore, by (7) and Ogura & Martin 
(2014, Lemma 1.1), we conclude that P{9k+i = j,Fk € 
G I Ml) = P{9k+i = j,Fk G G \ M 2 ), which is equiv¬ 
alent to (5). Also, the conditional probability (6) is in¬ 
dependent of k by (P3) and the time-homogeneity of 9. 
Therefore, is a discrete-time semi-Markov jump lin¬ 
ear system. 

The proof of the second statement can be done in the 
same way as the proof for the implication [2 ^ 3] of 
Ogura & Martin (2015, Theorem 12) due to (A2) and the 
assumption (1). Also, we can prove the third statement 
in the same way as the proof for [3 ^ 1] of Ogura & 
Martin (2015, Theorem 12) by (A2). The details of the 
proofs are thus omitted. 


Proposition 6 shows that the stability analysis of S could 
be reduced to the stability analysis of discrete-time semi- 
Markov jump linear systems. The next proposition gives 
a characterization of the exponential mth mean stabil¬ 
ity of discrete-time semi-Markov jump linear systems, 
extending Ogura & Martin (2014, Theorem 3.4) to the 
case where m is even. 

Proposition 7 Fori,j € {1, ... ,N}, letpij denote the 
transition probability of 9 from i to j. Assume that either 
m is even or is positive. Then, the following state¬ 
ments are equivalent. 

( 1 ) is exponentially mth mean stable; 

( 2 ) Ed is stochastically mth mean stable; 

(3) The (Num) x {Num) block matrixF with the (i,j)- 

block F, = PpE[F^'"^ I 9o = j,9i =i]G 
is Schur stable. 

For the proof of this proposition, we will need the next 
lemma. 

Lemma 8 Assume that m is an even integer. Let K he 
the closed convex hull o/(IR”)["*l = {xl™! : x G ffi"} in 
K"™. Then, there exists a norm ||| • ||| on and f G 

such that |||a;||| = f^x for every x G K x ■ ■ ■ x K. 


PROOF. We first show that AT is a proper cone (Tam 
& Schneider 2006, Chapter 26), that is, AT is a closed and 


convex cone having a nonempty interior and satisfying 

Kn{-K) = {0}, (8) 

where —K := {—x : x G K}. K is clearly a closed and 
convex cone. 

Let us show (8). For each i G {1, ■ ■. ,n}, let G denote 
the position of the monomial a;™ in the vector i.e., 
we assume that (x^"^^)£. = x™. Let us show the existence 
of a constant C > 0 such that every y G K satisfies 

n 

yii > 0, \yt.\ < (9) 

for all 1 < i < n and I < £ < Um. Take an arbi¬ 
trary y in the convex hull of (IR”)[™I. Then there exist 
xi,... ,xm G M" and positive numbers Ci,..., such 
that y = Without loss of generality we 

can assume cj = 1. Then, yi. = J2j=ii^j)T — 0 be¬ 
cause m is even. Next, let 1 < £ < Um be arbitrary 
and let a = (ai,..., a„) be the exponent of the mono¬ 
mial (xl™l)f, i.e., we suppose that (a;^™!)^ = v^a:". 
Then, the inequality of arithmetic and geometric means 
implies |(x[™l)^| < ■ Hence, the 

triangle inequality shows that \yi\ < A — 

TO”^V^Er=iEj^i(a;j)r < C'EEiyb) where C = 
m~^ maXc(-\/^). Therefore, every y in the convex hull 
of (IR”)[™1 satisfies (9). A limiting argument thus shows 
that (9) is satisfied by every y G K. Now assume 
y, —y G AT. This implies y£. = 0 for every i and therefore 
y£ = 0 for every £ by (9). Hence (8) holds. 

Finally, by Ogura & Martin (2014, Lemma 1.5), the dif¬ 
ference K — K = {x — y '. x,y G AT} equals the whole 
space M”™. This in fact shows that the interior of AT is 
nonempty because, in general, a closed and convex cone 
AT satisfying (8) has a nonempty interior if and only if the 
difference K — K coincides with the whole space (Tam 
& Schneider 2006, Chapter 26). 

Now, since AT is a proper cone, the Wdirect product 
AT X • • • X AT in is also a proper cone. Therefore 

there exists (Seidman, Schneider & Arav 2005, Section 2) 
a norm ||| • ||| on and a vector / e having 

the desired property. This completes the proof. 


We then prove Proposition 7: 


PROOF of Proposition 7. The case where E^ is pos¬ 
itive is shown in Ogura & Martin (2014, Theorem 3.4). 
Let us assume that m is even. We shall show the cycle 
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[1 ^ 2 ^ 3 ^ 1]. It is easy to prove [1^2], The impli¬ 
cation [2 =^> 3] can be proved in the same way as in the 
proof of Theorem 3.4 in Ogura & Martin (2014) without 
the positivity assumption. 

Let us prove [3 1] assuming m is even. Take a norm 

III • III on and / € satisfying the linearity 

property described in Lemma 8. By the equivalence of 
the norms on a finite-dimensional linear space, we can 
take a constant Ci > 0 such that Cf^ll'H < ||| • ||| < 
Cl 11 • 11. Let us consider the stochastic process , where 
Cl, ..., Cat denote the standard unit vectors in IR^. Us¬ 
ing the general identity ||a;['"l|| = HxH"* (see Parrilo & 
Jadbabaie 2008), we can show the inequality ||a;d(fc)||’” = 
\\xd{k)^^^\ = llegj^ (g) a;d(fc)[™l|| < Ci\\\ee^ Xd{k)^'^^\\. 
Since eg^ 0 a;d(fc)[’"l G K x ■ ■ ■ x K, the linearity of the 
norm ||| • ||| shows that 

E[\\xd{k)\r]<C^\\\E[eg,0Xd{km\ 

<CnE[eg,0Xd{km. 

On the other hand, by the identity 

CK+, 0 Xdik + 1)H] = j^E[eg, 0 Xdik)^^^] (11) 

proved in Ogura & Martin (2014, Proposition 3.8), if 
p{E) < 1 then there exist C 2 > 0 and /3 > 0 such that 

\\E[eg,0Xd{km<C2e-P'^\\eg,0x^^i 

= C2e-^'=||x[r'|| (12) 

= C2e-^'=||xo|r. 

Therefore, the inequalities (10) and (12) prove that 
is exponentially mth mean stable. This completes the 
proof of Proposition 7. 


Now we can readily prove Theorem 5. (Al) implies that 
either m is even or iSE is positive. Therefore, by Propo¬ 
sition 7 and the first statement of Proposition 6, we see 
that the following three properties are equivalent; the ex¬ 
ponential mth mean stability of 5E, the stochastic mth 
mean stability of and the Schur stability of A. This 
equivalence and also the second and third statements of 
Proposition 6 immediately prove the main result of The¬ 
orem 5. 

3 Applications 

In this section, we illustrate how Theorem 5, the main 
result of this paper, can be used to recover various stabil¬ 
ity characterizations of switched linear systems derived 
in the literature (Subsections 3.1-3.3), as well as to an¬ 
alyze the stability of systems that cannot be analyzed 
by current techniques in the literature (Subsection 3.4). 


Throughout this section, we will use the following no¬ 
tation (see Brockett 1973). For a matrix A G M"^", we 
define as the unique real matrix such 

that 

(exp(At))['"l = exp(A[„]t). (13) 


3.1 Regenerative switched linear systems 

In this subsection, we discuss some implications of 
the stability characterization provided for regenera¬ 
tive switched linear systems (Ogura & Martin 2015). 
We first recall the definition of regenerative processes. 
We say that a stochastic process cr is a regenerative 
process (Smith 1955) if there exists a random variable 
Ri > 0 such that (z) {crt+Ki}t>o is independent of 
{Wt}t<Ri,Ri}, and (a) has the same joint 

distribution as {(Tt}t>o- Then, following Ogura & Mar¬ 
tin (2015), we say that S given in (3) is a regenerative 
switched linear system if cr is a regenerative process. 

Let random variables be independent and iden¬ 

tically distributed and define Tk = Ri + - ■ ■ + Rk for each 
fc > 1. Then, a can be broken into independent and iden¬ 
tically distributed cycles {crt}o<t<Ti, {(Xt}T-,<t<T^, ■■■ 
(see Sigman & Wolff 1993). From this observation, it is 
immediate to see (((linlar 1975) that a is Markov regener¬ 
ative process with its embedded Markov renewal process 
being (0, r), where 9 is the constant sequence {1,1,...}. 
Therefore, we can define the exponential mth mean sta¬ 
bility of a regenerative switched linear system by Def¬ 
inition 3. We can furthermore prove, as a corollary of 
Theorem 5, the following stability characterization for 
regenerative switched linear systems originally given in 
Ogura & Martin (2015, Theorem 12): 

Corollary 9 (Ogura & Martin (2015, Theorem 12)) 

Let 'E be a regenerative switched linear system. Assume 
that (Al) is true and, moreover, there exists T > 0 such 
that Ri < T. Then, E is exponentially mth mean stable 
if and only if the matrix U[4>(i?i; 0)1™!] is Schur stable. 


PROOF. We first remark that, since Ri > 0, assump¬ 
tion (1) is satisfied. Moreover, Ri < T guarantees that 
(A2) is satisfied. Since (Al) is true by the assumption 
in the corollary, we can apply Theorem 5. Since 9 = 1, 
the matrix A given by (4) equals the Um x Um matrix 
PiiF;[$(ri;0)[™l I 9o = l,9i = 1] = F;[$(i?i;0)^]. 
Therefore, Theorem 5 readily completes the proof of the 
corollary. 


3.2 Semi-Markov jump linear systems 

We now consider another class of switched linear sys¬ 
tems called semi-Markov jump linear systems (Ogura & 
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Martin 2014). Assume that (6*,r) is a Markov renewal 
process taking values in {1,, N} x [0, oo), whose def¬ 
inition was given in the beginning of Section 2. Then, 
the stochastic process {(Jt\t>o defined by 

CTt = 9k, Tk <t < Tk+l, (14) 

is called a semi-Markov process (Qinlar 1975). Then, 
following Ogura & Martin (2014), we say that S given 
by (3) is a semi-Markov jump linear system if ct is a 
semi-Markov process. It is easy to see that ct is a Markov 
regenerative process with its embedded renewal process 
being (0,r). The mapping tt in (P2) is taken to be the 
identity. Without loss of generality, we can assume (1). 

The following corollary of Theorem 5, which extends 
Ogura & Martin (2014, Theorem 2.5) to not necessar¬ 
ily positive systems, immediately follows from (13) and 
(14): 

Corollary 10 Suppose that is a semi-Markov jump 
linear system and assume that conditions (Al) and 
(A2) hold. Then, Y is exponentially mth mean stable 
if and only if the (Num) x (Num) Teal block matrix 
A = with the (i, j)-block Aij being defined 

by Aij = Pji£;[exp((Aj)[„]Ti) | 6»o = j,6»i = i] is Schur 
stable. 

3.3 Markov jump linear systems 

In this subsection, we show that Theorem 5 can recover 
stability characterizations (Ogura & Martin 2014, Fang 
& Loparo 2002) for a class of switched linear systems 
called Markov jump linear systems (Costa et al. 2013). 
We say that Y given by (3) is a Markov jump linear sys¬ 
tem if CT is a time-homogeneous Markov process. Gen¬ 
eralizing the stability characterizations of Markov jump 
linear systems in Fang & Loparo (2002, Theorem 3.3) 
for m = 2 and in Ogura & Martin (2014, Theorem 5.1) 
for positive systems, we can show the next corollary of 
Theorem 5: 

Corollary 11 Suppose that Y is a Markov jump linear 
system and a can take values in{l,..., N}. Assume that 
either m is even or Ai, ..., An are Metzler. Let Q be the 
infinitesimal generator of Markov process a. Then, E is 
exponentially mth mean stable if and only if the matrix 
By: = Q"' ® In„, + is Hurwitz stable. 

PROOF. Notice that, for > 0 arbitrarily chosen, 
CT is a Markov regenerative process with the embedded 
Markov renewal process {0,t) = {{crkh, kh)}k>o by the 
Markovian property of ct. Therefore, S is a Markov re¬ 
generative switched linear system. The assumption (1) 
is trivially true and, also, condition (A2) is satisfied 
with T = h. Moreover, the assumption of the corol¬ 
lary ensures that condition (Al) also holds true. Let 


Pij{h) = P{9i = j \ 9o = i) = P{(Th = j \ <70 = i). Then, 
by Theorem 5, E is exponentially mth mean stable if and 
only if the block matrix A{h) = [A{h)ij]i<ij<:N given 
hyA{h)ij =Pji(/i)A[$(/i;0)[’”l | au = i,<Jo is Schur 
stable for every h > 0. We continuously extend the do¬ 
main of A to the origin by letting A(0) = I. The conti¬ 
nuity of the extended A at the origin can be verihed by 
the Dominant Convergence theorem. Then, to complete 
the proof of the corollary, it is sufficient to show that 

A{h) = exp{BYh) (15) 

for every h > 0. 

To prove (15), we need to show that 

A{h)A{h') = A{h + h') for all h,h' >Q (16) 

and 

A!{t)) = BY. (17) 

Let us show (16). Notice that, by (11), the trajectory x 
of E satisfies 

E[ec,t^h ® = A{h)E[e„t G a;(t)f™'] (18) 

for all t > 0 and h > 0. This in particular im¬ 
plies A{h h'){eao G a;[("') = A{h)A{h'){eao G ajQ™') 
for all CTo G {1,...,A} and a;o € M". Since the set 

: i € {1..., IV} and xq € M"} spans the whole 
space by (Ogura & Martin 2014, Lemma 1.4), we 

obtain (16). 

Then let us show (17). We compute AC(0), the {i,j)- 
block of A'(0), for each pair (z,j). First assume i 
j. Since limh^o E[^{h;0)^'^^ | cr/i = i.ctq = j] = I, 
by the Dominated convergence theorem, we can show 
Aij{0) = lim/j_j.o(pji(Ii)/ — 0)1 h = Xjil. Next assume 
i = j. Let 7 denote the number of transitions of ct on 
the interval [0, h] when ctq = i. Then the event {ah = 
i,CTo = t ,7 = 1} has probability zero. Moreover, using 
the big-0 asymptotic notation, we can show that 

A[$(/i;0)[™l \ ah = i,ao = i,y = f] 

g(^i)[m,]^, for £ = 0, 

I -\- 0{h), for £>2, 

P{'j = 0) = and P (7 > 2) = 0{h^). Therefore 

Au(h) - Au{0) _ puih)eOAi^A - I ^ 

h h + y > 

{Ai)[m] + Xiil 

as /i —> 0. The above argument proves (17) and, there¬ 
fore, completes the proof of the corollary. 
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3-4 Markov jump linear systems with periodic mode ob¬ 
servation 

In this subsection, we present an example of a sys¬ 
tem to which none of the results in Corollaries 9-11 
are applicable. Consider the constant-gain state- 
feedback control of the Markov jump linear sys¬ 
tem dx/dt = AprfXlt) BprtUlij), where Api, 
Ap^N G K”""”,’ Bp^i,...,Bp^N G and’r 

is a time-homogeneous Markov process with state 
space {1,... ,N} and infinitesimal generator Q. If the 
controller can measure r at any time instant, one can 
consider the feedback control 

u = KrX (19) 

with mode-dependent gains Ki G (i = 1,..., TV). 

It is well known (Costa et al. 2013) that, under this ideal 
situation, one can find feedback gains Ki that stabilize 
the closed-loop system in the mean-square sense by solv¬ 
ing linear matrix inequalities. 

Following the formulation of Cetinkaya & Hayakawa 
(2015) in discrete-time, we here consider a more realis¬ 
tic situation where only the periodic samples {rkh\k>o 
with a known sampling period h > 0 are available to 
the controller. Precisely speaking, we assume that the 
feedback control takes the form 

U = KqX (20) 

where the stochastic process q is defined by qt = r^h if 
kh < t < {k-\- l)h for all t > 0 and k > 0. We emphasize 
that, though we assume that only periodic samples of r 
are available, the infinitesimal generator Q is assumed to 
be known before the feedback control is designed. Defin¬ 
ing a = {r,q) and Aa- = Ap^j. + BprKq, we can write 
the closed-loop equation in the form (3), which we de¬ 
note by E/j. Since r is time-homogeneous and Marko¬ 
vian, we can see that cr is a Markov regenerative process 
with an associated embedded Markov renewal process 
{irkh,kh)}k>o- The function tt in (P2) maps the pair 
ihj) G {1,..., iV}^ to i. Then, using Theorem 5, we can 
prove the following characterization of the exponential 
mean stability of E/ji 

Corollary 12 Assume that m is even. For each j G 
{1,.. .,N}, letBj = Q^(8)/n„+©j^i(Glpj))[m]. Define 

N 

Ah = Y^ exp{Bjh){{ejeJ) 0 /„„). (21) 

i=i 

Then, E/j is exponentially mth mean stable if and only if 
Ah is Schur stable. 


PROOF. Since m is even, condition (Al) is satis¬ 
fied. Moreover, because Tk+i — Tk = h, condition (A2) 


as well as (1) hold true. Therefore, by Theorem 5, it 
is sufficient to show that the matrix A given by (4) 
equals Ah. To compute Aij, observe that cr confined in 
[0, h) with the condition vq = j is a time-homogeneous 
Markov process with state space {(1, j),..., {N,j)} and 
infinitesimal generator Q. Therefore, from (15), Aij 
equals the (i,j)-block of the matrix exp{Bjh). Hence, 
the j-th block-column of A equals that of exp{Bjh). 
This proves A = Ah, as desired. 


Remark 13 Note that we can use neither Corollaries 9 
nor 10 to analyze the stability of E/j. First, it is shown 
in Ogura & Martin (2015, Example 7) that a cannot 
be a semi-Markov process. Moreover, though a can be 
realized as a regenerative process, the realization in Ogura 
& Martin (2015, Example 7) does not satisfy (A2) by the 
following reason. In the embedded renewal process {0, t) 
of the realization, the renewal times {Tk}k>Q are given 
as the sampling times t = kh (k = 0,1,...) such that 
r{t) = vq. However, the difference Tk+i — Tk can take an 
arbitrary large number and, hence, cannot be bounded by 
a uniform and finite number. 


Let us see an example. Consider the Markov jump linear 
system with the following coefficient matrices: 



1 

O 

o 

0 


1 

0 

0 

» 1 

Al = 

0 -0.545 0.626 

) A 2 — 

0 -0.106 0.087 


0 -1.570 1.465 


0 -3.810 3.861 


1,80 

-0.3925 

4.52 


^3 — 

3.14 

0.100 

-0.28 

5 


-19.06 -0.148 1.56 


0 


0 


-0.064 

-0.283 

, B2 = 

0 

, B3 = 

0.195 

0.333 


0.087 


-0.080 


and the infinitesimal generator 


Q 


-0.53 0.32 0.21 

0.50 -0.88 0.38 
0.40 0.13 -0.53 


This system, taken from Blair & Sworder (1975), models 
a certain economic system. We denote by Eq the closed- 
loop system when the classical feedback control (19) is 
applied to this system. In Costa et al. (2013), the feed¬ 
back gains that stabilize Eq (in the mean square sense) 
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0 0.1 0.169 0.2 0.3 

h 

Fig. 1. h~^ log(p(^?i)) versus h 

with the minimum norm are obtained as 

Ki = 2.0343 14.5181 -23.5917 , 

K2 = 1.0187 73.0961 -78.7596 , 

TsTs = 93.6651 -11.4921 11.6875 • 

We use Corollary 12 to investigate how the stability 
property of Sq is altered when the feedback control (19) 
is replaced with the one in (20) based on periodic 
observations. For m = 2 and period h = (0.001)£ 
{£ = 1,..., 300), we compute the spectral radius of Ah 
given in Corollary 12. In Fig. 1, we show the graph 
of hr^ \og{p{Ah)) as h varies. From the corollary and 
the graph, we determine that the system Yih is mean 
square stable if and only if 0 < /i < 0.169 [years]. This 
implies that, to guarantee the stability of the controlled 
economic system, r must be sampled with a period less 
than about 2 months. It is interesting to observe that, 
as /i —>■ 0, the quantity h~^ log(p(^/i)) becomes close to 
—0.250, the maximum real part of the eigenvalue of the 
matrix that characterizes the stability of the origi¬ 
nal system Sg by Corollary 11. This shows that, in the 
limit of —> 0, the stability of the original closed-loop 
system Sg is “recovered” by Y^h- 

In Figs. 2a and 2b, we show 100 sample paths of ||a;(t)|p 
and their sample averages when /i = 0.1 and h = 0.3, 
respectively. For the generation of each sample path, we 
take xq randomly from the unit sphere in and rg from 
the uniform distribution on {1,2,3}. We can see that 
mean square stability is achieved with sampling period 
= 0.1, while the closed-loop system exhibits instability 
for h = 0.3. Finally, for each value of h, we show sample 
paths of CT = (r, q) in Fig. 3. We notice that the change of 
the values of q occurs only at sampling instants, shown 
by the dotted lines. 



t 


(b) h — 0.3 (unstable) 

Fig. 2. Sample paths of ||a;(t) |p (thin lines) and their averages 
(thick lines) for h = O.l and 0.3 



0 12 3 

t 


(a) h — 0.1 (stable) 



0 12 3 

t 


(b) h = 0.3 (unstable) 

Fig. 3. Sample paths of cr = (r, q) for h = 0.1 and 0.3. Solid 
lines, dashed lines, and dotted lines show r, q, and sampling 
instants, respectively. 





































4 Conclusion 

In this paper, we have investigated the mean stability of 
Markov regenerative switched linear systems. The class 
of switched linear systems contain a wide variety of im¬ 
portant stochastic switched linear systems that have ap¬ 
peared in the literature. We have shown that the mean 
stability of a Markov regenerative switched linear sys¬ 
tem is characterized by the spectral radius of a matrix 
arising from its transition matrix. A numerical example 
was presented to illustrate the obtained result. 
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